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Abstract 


An overview of the Joint Agency Commercial Imagery Evaluation (JACIE) team 
is presented. JACIE, composed of the National Aeronautics and Space Administration 
(NASA), the National Imagery and Mapping Agency (NIMA), and the U.S. Geological 
Survey (USGS), was formed to leverage government agencies' capabilities for the 
characterization of commercial remote sensing data. Each JACIE agency purchases, or 
plans to purchase, commercial imagery to support its research and applications. It is 
critical that the data be assessed for its accuracy and utility. Through JACIE, NASA, 
NIMA, and USGS jointly characterized image products from Space Imaging’s IKONOS 
satellite. Each JACIE agency performed an aspect of the characterization based on its 
expertise. NASA and its university partners performed a system characterization 
focusing on radiometric calibration, geopositional accuracy, and spatial resolution 
assessment; NIMA performed image interpretability and feature extraction evaluations; 
and USGS assessed geopositional accuracy of several IKONOS products. The JACIE 
team purchased IKONOS imagery of several study sites to perform the assessments and 
presented results at an industry-government workshop. Future plans for JACIE include 
the characterization of DigitalGlobe’s QuickBird-2 image products. 
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1. Introduction 


As operating budgets in government agencies continue to shrink, improved ways 
of doing business must be explored. The purchase of commercial imagery to support 
Earth Science research, civil government applications, and defense-related intelligence 
gathering is a cost-reduction method currently under investigation. Strategic partnerships 
that capitalize on common goals can also prove beneficial. Both NASA and the National 
Imagery and Mapping Agency (NIMA) have significant data purchase programs in place 
with the commercial sector. The U.S. Geological Survey (USGS) is also planning to 
implement data purchase activities. Because these agencies experience common issues 
dealing with the purchase of commercial imagery, a Joint Agency Commercial Imagery 
Evaluation (JACIE) team was formed. The JACIE team leverages capabilities and 
provides a unified government voice in interfacing with industry. The purpose of this 
paper is to describe JACIE activity and its significance to U.S. government research and 
applications. 

2. Background 

In 1 997, the NASA Earth Science Enterprise (ESE) initiated the Scientific Data 
Purchase (SDP), a $50 million project to ascertain the utility of commercial remote 
sensing datasets for Earth Science research and applications. Through the SDP, NASA 
purchases commercial remote sensing data products from five data providers: Earth 
Satellite Corporation (EarthSat), Positive Systems, Inc., DigitalGlobe (formerly 
EarthWatch)/Intermap Technologies, AstroVision, Inc., and Space Imaging, Inc. The 
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NASA Stennis Space Center (SSC) Earth Science Applications (ESA) Directorate 
manages the SDP and is responsible for the verification and validation of the delivered 
data sets. For a more detailed description of the SDP, access the project Web page at 
http://www.esad.ssc.nasa.gov/datapurchase. 

NIMA established the Commercial Imagery Program (CIP) in 1998 to support the 
acquisition and exploitation of commercial imagery by NIMA and its Department of 
Defense (DoD) customers. The CIP purchases imagery products from Space Imaging, 
SPOT, Digital Globe (Earthwatch), and Star3I. Imagery purchases through the CIP also 
include license upgrades so the data can be shared by all DoD/Intelligence activities. CIP 
also purchases Landsat-5 and Landsat-7 data from the USGS EROS Data Center (EDC). 
Assessments of Image Quality & Utility (IQ&U) for commercial sources of imagery are 
performed by the Civil and Commercial Applications Project (CCAP), part of NIMA’ s 
Imagery Support and Assessments Branch (ASAI) (NIMA, 2000). 

The USGS expects to increase its use of commercial geospatial data in its science 
and mapping programs. In this regard, the USGS Land Remote Sensing Program is 
actively pursuing the development of contractual mechanisms and agreements to 
facilitate cost effective access to a wide range of commercial off-the-shelf remote sensing 
data sources. The USGS is also interested in validating the accuracy and utility of such 
sources - both U.S. and foreign. Planned contracts would be available for optional use by 
other civil agencies to help meet their commercial imagery needs. This work is the 
continuation and expansion of a USGS historical and long standing service to the civil 
community. 
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Since Space Imaging, Inc., was the first company to successfully launch a 
commercial high-resolution remote sensing system, there is a great deal of interest by 
NASA, NIMA, and USGS in understanding IKONOS' utility for science research and 
civil and defense-related applications. Commercial data products must be highly 
characterized to be useful to the government’s science and applications community. 
However, commercial providers may not characterize systems as science researchers 
desire because commercial and science requirements often differ. This is a new paradigm 
for the government’s imagery user community - a community historically having 
significant insight into sensor design and operation. Additionally, government-built 
satellite systems cannot compete with commercial industry. Thus, commercial providers 
must fill government requirements for high-resolution data. This new way of doing 
business, therefore, requires independent data characterization. 

As part of its SDP role, NASA ESA Directorate conducted an extensive effort to 
independently characterize the performance of the IKONOS system. Recognizing that 
NIMA also procures IKONOS data, and that USGS is planning to procure IKONOS data, 
the three agencies formed the JACIE team in February 2000. The team consists of the 
three government agencies and several university affiliates. The purpose of the group is 
to capitalize on mutual interests and to leverage resources for the characterization of 
commercial remote sensing data. A JACIE Interagency Space Act Agreement was 
signed by the three founding agencies in June, 2002. 


3. JACIE Activities 


Each JACIE team member brings different strengths to the characterization 
activity, reducing the cost of a full evaluation by minimizing duplication of effort by the 
government and industry. USGS, having a long history in cartography, conducts 
IKONOS geometric accuracy assessments. This includes evaluation of IKONOS 
georectified, orthorectified, digital elevation models, and stereo image products. NIMA 
performs DoD and intelligence-based product evaluations, including image 
interpretability, feature extractions, and photogrammetry assessments. NIMA's customers 
also evaluate the utility of IKONOS data for DoD and intelligence use. NASA performs 
overall system performance characterization by evaluating SDP products against contract 
specifications for spatial resolution, geometric accuracy, and radiometric accuracy. 
Additional NASA efforts include evaluations of IKONOS data processing techniques and 
comparisons with other systems, such as Landsat-7 and MODIS. NASA has partnered 
with experts from the Earth Science community to support system characterization. These 
experts from South Dakota State University, the University of Arizona, and the 
University of Maryland have years of experience in characterizing and validating NASA- 
developed sensors. Each agency’s customers also provide a valuable role in the 
characterization effort as they use IKONOS data to support science research and 
applications. The roles of each JACIE team member are illustrated in Table 1. The table 
indicates that multiple team members independently perform each aspect of the 
characterization effort, thus strengthening the overall result. JACIE team members have 
worked together to plan IKONOS data acquisitions, to conduct field measurement 
campaigns, and to analyze IKONOS imagery. 
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JACIE data evaluation results have been produced for each aspect of the 
characterization, and several key improvements to IKONOS data quality have resulted. 

In terms of radiometric characterization, JACIE-led vicarious calibration efforts 
determined that Space Imaging's radiometric calibration coefficients were inconsistent 
with those produced by the JACIE team (Pagnutti, 2001). The JACIE team collaborated 
with Space Imaging to investigate the inconsistency. As a result, the JACIE-developed 
radiometric calibration results were incorporated into an updated set of Space Imaging 
calibration coefficients (Peterson, 2001). 

Regarding spatial resolution, Space Imaging applies an image restoration 
algorithm, called Modulation Transfer Function Compensation (MTFC), which 
“sharpens” the imagery and improves feature identification. This results in an effective 
enhancement to the spatial resolution. Through JACIE investigations, it was discovered 
that the MTFC algorithm, or kernel, was incorrectly applied by Space Imaging, creating 
an over-compensation in the image cross-track direction, and under-compensation in the 
along-track direction. Space Imaging subsequently rotated the MTFC kernel to correct 
the error. 

As part of a collaborative effort undertaken by NIMA, USGS, and Space Imaging, 
a comparison of Space Imaging’s rigorous sensor model and an adjustable form of the 
rational polynomial coefficient (RPC) model was conducted through a large area block 
adjustment evaluation. The objective was to determine if the adjustable RPC model 
implemented by NIMA could be used to triangulate multiple IKONOS stereo strips to 
NIMA mapping accuracy specifications. Test cases included individual stereo pairs as 
well as triangulated blocks of overlapping strips. Image strips were triangulated with 
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varying numbers of ground control points. During the course of the evaluation, it was 
revealed that a 2-pixel bias on the western edge of the stereo strips was producing a 
cumulative error in the block adjustment (Ager, Bresnahan, 2002). This error was 
investigated by Space Imaging and subsequently corrected, and Space Imaging has been 
able to perform a similar block adjustment that exceeded NIMA’s accuracy requirement 
(Dial, 2002). The detection of this pixel bias, the open discussion of its possible causes 
and corrections, and Space Imaging’s response to its evaluation partners at NIMA and 
USGS to remedy the error occurred within the JACIE mechanism invoked by the 
principal investigators at NIMA and the USGS. 

The USGS evaluated digital elevation models (DEMs) produced from IKONOS 
stereo pairs and found that the vertical accuracy of the DEMs, in some cases, exceeded 
the expected error limit of seven meters. A DEM over one study site revealed a 
systematic vertical bias of about 15 meters — double the expected limit. As a result of 
this finding. Space Imaging changed its DEM production procedures to require more than 
one stereo-image pair in the DEM block adjustment. (Constance, 2002) 

4. NASA, NIMA, and USGS IKONOS data user contributions 

NASA's Earth Science community has also played a key role in the data 
characterization effort. As SDP customers, scientists assess the performance of IKONOS 
data in various research areas. IKONOS data has supported research in precision 
agriculture (Seelan, 2001), coral reef assessment (Andrefouet, 2001), Antarctic ice shelf 
investigation (Bindschadler, 2001), urban environments (Small, 2001), and many other 
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areas. In many cases, scientists uncovered data issues that were detected only by 
performing specific scientific studies. 

Likewise, NIMA customers within the DoD conduct defense and intelligence 
applications using IKONOS data. Uses of IKONOS imagery by NIMA include support 
to civil authorities in disaster response (Sidor, 2001), updating theater and tactical maps 
for U.S. Armed Forces (NIMA, 2001), and the production of specialized imagery and 
map products for the U.S. Naval fleet and special forces (Ecuyer, 2001). 

Within the USGS, the use of IKONOS data has significantly contributed to the 
agency’s National Land Cover Characterization Project. Through use of IKONOS data 
provided by NASA SDP and other high-resolution data, an approach was developed to 
quantify impervious surfaces and forest canopy density. The practicality and 
affordability of the proposed method for large-area mapping was tested over several 
geographic areas with successful results. The approach has been implemented and 
recommended for mapping two ecologically important parameters for the entire United 
States. The availability of the IKONOS data acquired form diverse biome and ecoregions 
are especially valuable for training/testing of land cover algorithms and mapping 
processes where other high-resolution imagery is not available or of only low quality 
(Yang, 2002). In another USGS application, Chen et al. (2002) used IKONOS 
panchromatic and multispectral imagery, Landsat 7 imagery, and airborne scanning 
LIDAR to estimate forest structure and stand density in the Black Hills of South Dakota. 
The results of this study indicated that IKONOS data can serve to make the important 
distinctions between tree canopy coverage and exposed under story grasses near peak 
summertime greenness (Chen et al., 2002). 
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Results of the JACIE characterization efforts have been presented at an annual 
JACIE High Spatial Resolution Commercial Imagery Workshop. The purpose of the 
annual workshop is to communicate the results of the JACIE characterization effort and 
to provide a forum for data user and industry interaction. The workshop is the only one of 
its type to focus on the use of commercially purchased high spatial resolution remote 
sensing data within the government. IKONOS characterization results and government 
applications were presented at the 2001 workshop held in Greenbelt, MD, and the 2002 
workshop held in Reston, VA. More than 150 representatives from NASA, NIMA, 
USGS, and the agencies’ user communities attended each workshop. Representatives 
from Space Imaging also participated in the workshops and presented results of the 
company's internal IKONOS calibration efforts. Feedback from workshop participants 
and Space Imaging indicated that the meetings were a success and that similar workshops 
should continue to be held in the future. 

5. JACIE Benefits 

In addition to the IKONOS characterization efforts, the JACIE team has produced 
several other benefits. The group provides a single government interface to Space 
Imaging, not only for characterizing the IKONOS system, but also for obtaining system 
information from Space Imaging. Through JACIE technical interchange meetings, 
several features of IKONOS data were uncovered. For instance, the JACIE team effort 
resulted in the discovery that the IKONOS system compresses datasets onboard the 
spacecraft. Space Imaging has stated that the compression is lossless. In addition, the 
compression minimally impacts research because the 1 1-bit data provide an increased 
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dynamic range — almost an order of magnitude better than existing 8-bit systems. The use 
of MTFC described above was another aspect uncovered through JACIE interactions 
with Space Imaging. The effect of the MTFC process has been investigated, and results 
were presented at the annual JACIE workshop. Through JACIE interaction, NASA also 
learned that NIMA purchases certain data products from Space Imaging that were not 
available to NASA. As a result, NASA modified the Space Imaging SDP contract so that 
these new products could be made available to NASA's Earth Science customers. 

The government-industry relationship developed through JACIE played a key role 
in the success of the JACIE effort. It is important to note that the JACIE effort is not 
intended to discredit industry or cast suspicion about a commercial provider’s ability to 
provide high quality data. Rather, the joint government activity to conduct independent 
assessments helps the government be a smart buyer of the data for its users. Any 
discrepancies in IKONOS data uncovered through JACIE efforts were discussed openly 
and constructively with Space Imaging. The interactions between Space Imaging, 

NASA, NIMA, and USGS established a spirit of trust and cooperation toward improving 
IKONOS data quality for all users. 

The JACIE team provides a valuable service to both the remote sensing industry 
and the government. By performing independent evaluations of commercial systems, the 
JACIE team provides a non-biased assessment of commercial data. These assessments 
can only help the industry to gain credibility in both government and non-government 
markets. 

Data characterization is of vital importance to the government. NASA scientists' 
use of commercial data can potentially influence global change decisions and public 
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policies. NASA must ensure that the delivered data products possess the accuracies 
promised by the data provider. It is imperative that these sources of data be well 
characterized in well-understood ways so they are used appropriately. 

Likewise, NIMA supports the DoD intelligence community through the 
distribution and management of remotely sensed data. Commercial data is used by 
NIMA users to support intelligence and mapping tasks that are driven by 
NIMA/DoD/Intelligence requirements and standards. Commercial imagery can best be 
used to answer an intelligence problem if some idea of its image quality and utility is 
known. This is typically a quantification of imagery interpretability as expressed through 
a National Imagery Interpretability Rating Scale (NIIRS) (Pike, 1998), a form of 
assessment not normally performed by either the civil community or the vendor. 
Similarly, commercial imagery intended for the production of maps must meet NIMA 
accuracy standards, requiring an independent verification of accuracy. Thus, it is critical 
that commercial data be independently characterized to ensure that defense and 
intelligence decision makers use accurate and reliable information. 

The USGS also has a vital interest in understanding the qualities and 
characteristics of the imagery that it applies in its scientific investigations and mapping 
products provided to the nation. Furthermore, the many government agencies, both 
Federal and State, rely upon the USGS for the assessment of commercial products for 
their mapping applications. All of the above-mentioned uses of IKONOS data strongly 
justify the need for an independent commercial data characterization effort. 

6. Future Role of JACIE 
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In addition to IKONOS, the JACIE team also plans to evaluate other commercial 
satellite systems as they become operational. The next system of primary interest to 
JACIE characterization is the DigitalGlobe QuickBird-2 satellite. This system provides 
0.6-meter panchromatic and 2.44-meter multispectral imagery, and represents the best 
non-classified spatial resolution available from space. Other potential roles for JACIE 
include the following activities: 

■ Develop data requirements by interacting with user communities 

■ Research and develop new data validation techniques 

■ Develop standard characterization methods and procedures 

■ Establish industry guidelines and standards for procuring remotely sensed imagery 

■ Support the development of data policies 

■ Design and implement industry-government workshops 

7. Summary 

The NASA, NIMA, and USGS JACIE team was established in 2000 and 
formalized in June, 2002, with a goal of performing independent data characterization for 
commercial data used by the three agencies. The team has collaborated to characterize 
the 1 -meter panchromatic and 4-meter multispectral image products available from Space 
Imaging, Inc.’s IKONOS satellite. Characterizations focused on radiometric accuracy, 
spatial resolution, and geopositional accuracy assessments. The JACIE effort and 
collaborative interactions with Space Imaging have resulted in several improvements to 
IKONOS data quality. NASA, NIMA, and USGS data users have performed a variety of 
research and applications using IKONOS data. JACIE characterization and user research 
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results were presented at JACIE-sponsored workshops in 2001 and 2002. Focus for 
JACIE will also include characterization of DigitalGlobe’s QuickBird-2 high-resolution 
panchromatic and multispectral data. 
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Table 1: JACIE Characterization Team Members and Roles 


Organization 

Spatial 

Geometric 

Radiometric 

Applications 

USGS 


X 



NIMA 

X 

X 

X 

X 

NASA Stennis 

X 

X 

X 

X 

University of Arizona 

X 


X 


South Dakota State University 

X 

X 

X 


University of Maryland 




X 

Government User Community 



X 

X 

Space Imaging 

X 

X 

X 
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